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ABSTRACT 



Title: An Experimental Evaluation of Programming Student 

Study Behavior in Undergraduate Courses. 

The purpose of this research project was to examine 
a programmed study procedure employing behavior theory con- 
cepts in teaching undergraduate course material. 

The procedure involved constructing study units con- 
taining a specific reading assignment and study guide ques- 
tions. When the student felt he had mastered the material 
he was given a readiness test. While this test was not 
given a grade, he had to achieve a specified performance 
criterion (usually 80 percent correct) in order to progress 
on through the course- If lie did n.ot achieve this criterion 
he was allowed to rcstudy the unit and to take another readi- 
ness test. Progress through t lie course was contingent upon 
satisfactory performance, 

The effects of such a procedure were assessed in terms 
of (1) performance on hourly exams, (2) degree of participa- 
tion in the program, and (3) student attitude toward programmed 
course. The project involved (1) t li c refinement of pro- 
grammed study technique by examining parameters suggested as 
relevant and (2) the testing of the procedure's range of 
usefulness in other course settings. Thirteen courses involv- 
ing approximately 850 undergraduate students and six faculty 
members over an 18 month period were involved in this inves- 
tigation. 

The results of the investigation indicated that the 
programmed study procedure produced significantly better 
performance in all courses taught within the Department of 
Ps ycno lo gy and in about a third of non- psychology courses. 
Participation was highest in those courses which required 
the student to complete all units before gaining access to 
the hourly exams, Student attitude was generally favorable 
toward the programmed study procedures, they recognized the 
added mastery achieved by the new instructional technique. 

The programmed study technique was viewed by all involved as 
a valuable educational technique in the small college setting. 
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This project grew out of a norc informal effort by the 
authors to apply the instructional technique used by Keller 
at Arizona State University and discussed in bis article, 

"Good bye , Teacher", appearing in the Journal of Aonl ied 
behavioral An a 1 ys i s (&) . 

Mot reported, but herewith mentioned, arc the experiences 
of the autliors 1 attempt at educational research after careers 
in laboratory research. Two tilings have impressed then: 

(1) the difficulty in maintaining rigorous experimental con- 
trol when the manipulation of experimental parameters arc 
placed in the hands of others, and (2) details of obvious 

i 

importance to the psychologist do not necessarily have high 
face validity to non- psychologis ts . 

The authors wish to express their gratitude to those who 
have encouraged, assisted, challenged and reinforced their 
research efforts. First, to some 850 undergraduate students 
at Bridgewater College who have participated in programmed 
study courses. In years of lecturing we have never had such 
an opportunity to know so many students so well, This alone 
almost mal.es it worthwhile to give up that interesting pastime 
cal led lecturing, 

A special word of appreciation must be given to col- 
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leagues who volunteered their courses for inclusion in this 
study, while maintaining their composure and good humor at our 
necessary interference with their courses: Professor Killian 

Barnett for his courses in Principles of Sociology, Prof- 
essor linmcrt Bittingcr for his courses in Cultural Anthro- 
pology, Professors David Tetzler and William Willoughby 
for their courses in Religions of the "World* 

Finally wc wish to thank Dean Dale V* Ulrich for 
providing an academic setting in which research is possible, 
even when it runs contrary to his initial pred i lcct ions t and 
when it leads to vocal objections from faculty colleagues and 
anxious s tudents . 
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INTRODUCTION 



This is an era of reforn and change in education. After 
five years of widespread student dissent, the campus rood is 
changing fron rage to reforn. l!ost educators arc in agrccrcnt 
that now is the tine to institute new reforms, formalize exper 
incuts, and strengthen what has already been done. 

Educational institutions arc being challenged to im- 
prove the quality of instruction and the efficiency with 
which they arc able to provide this instruction. A recent 
Carnegie Foundation report (7J has suggested reducing the 
amount of tine required for higher education degrees. Such 
a suggestion behooves educators to take a close look at the 
quality and efficiency of their teaching. 

In the past decades numerous innovations have been made 
with varying degrees of success in different locals. Closed 
circuit 'IV, educational TV, and now public TV (as "fesnre 
Street”) have been used to bridge the gap between available 
resources and quality education for expanding numbers of stu- 
dents. Team teaching has been seen as a way to deal effective 
ly with increasing number; of students in the classroom pro- 
viding an individualized educational experience for nany stu- 
dents in a way which would not increase the demands for 
larger professional staffs. 

In the past decade psychologists have been perfecting 

7 
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t lie tcchnologica 1 aspects of their discipline, and have made 
interesting and provocative appl ica t ions of this technology 
in the schoolroom, clinic, hospital and prison. It should 
not be surprising that psychologis ts should turn their atten- 
tion to practicing what they preach by applying their technology 
in the teaching of their own courses. Many psychologists com- 
mitted to a behavioral technology have started applying such 
a technology to their own teaching. llovcvcr, t he number sub- 
jecting tliis effort to experiment al scrutiny is, so far, pain- 
fully small. 



Programmed Instruction vs ( Programmed Study ; 

educational literature is rich with references to teach- 
ing machines and programmed instruction. In programmed in- 
struction -- whether it be the brandling program of Crowder 
( 13 ), or the linear program of Skinner (11), or a program for a 
teaching machine or a programmed text -- essentially what is 
done is to prepare an orderly sequence of stimulus material to 
be learned as dictated by the programmer's theoretical commit- 
tment. Provision is made for reinforcement by information 
feedback relative to the correctness of the student's response, 
Perhaps the most frequently encountered problem faced by 
the instructor is the dirtli of available programs which greatly 
restricts his freedom of choice as compared with 1 he selection 
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among standard textbooks, 
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In this report a procedure for use with any standard text- 
book will be discussed. The term PROGRAMMED STUDY is used 
in contradistinction to PROGRAMMED INSTRUCTION, Programmed 
instruction involves primarily the arrangement of stimulus 
materials, the presentation of such materials, and consequent 
feedback for reinforcement* In programmed study the emphasis 
in upon the management of the student’s behavior when studying 
regular course material (sue]) as a textbook assignment) , This 
is done by breaking the assigned material down into ’'study 
units" from 10-30 pages, and then managing the contingencies 
controlling the student’s progress through this set of study 
units, While it is possible to employ programmed text mater- 
ials, regular standard textbooks have more frequently been em- 
ployed, This feature permits a wide range of application and 
great freedom of choice among available materials by the in- 
structor. 

Programmed Study : 

Pres sy (112) has said that the reason his "testing and 
teaching" machine of the did not catch the fancy of the 

educator was that at that time there was no satisfactory the- 
oretical structure to guide the preparation of stimulus mater- 
ials composing the program, B- V, Skinner lias since provided 
this theoretical structure in his concept of operant behavior, 
and has pioneered in the development of programmed instruction, 

<) 
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His attention has been directed primarily toward the programmed 
textbook. 

The person who lias exerted the greatest influence on the 
application of operant bciiavior concepts to the management of 
student study bciiavior has been Fred Keller. 

Kcl ler f s Procedure i 

In 1 Q G 3 Keller (6.) started a programmed study procedure 
at Arizona State University which lie first used during World 
War II while teaching Norse code signalling to armed forces 
personnel. He noticed that trainees could train themselves 
if given appropriate feedback lie then took his method to the 
University of Brazil in l'dGIh While there, a political up- 
heaval at that university initiated his move to Arizona State 
University, Here he continued to develop his new educational 
procedure , 

One of the important appects in Keller s system was to 
have tiic student assume an active role in t lie educational pro- 
cedure. He stated that a student would acquire much more from 
the educational environment if lie took an active role in this 
process rather than a passive role in which he did nothing more 
than listen to materials presented by a lecturer. 

A second requirement was that the student achieve a 
level of complete mastery in the subject natter before roving 
on to new material, To rake it possible for all students to 
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do this, Keller provided a program of instruction which was 
graded in steps which made the probability of success very 
high for those students who studied the material a legitimate 
amount of time. 

A third important aspect of his program was that of pro- 
viding immediate feedback (reinforcement) for cadi student con- 
cerning his understanding of the material This meant that 
all quizzes and tests that were administered to the student 
were graded immediately, allowing the student to know at what 
level lie had performed on that particular material* 

A fourtli point which Keller felt important to the main- 
tenance of student study behavior was to allow' the student to 
pace himself in his program of study. He felt that this free- 
dom to work rapidly when so inclined and to slack off when other 
demands were pressing made t lie educational environment much 
more rewarding for the student. 

Finally, Keller stressed the importance of the student 
proctor. He felt that the advanced psychology student was 
capable of relating to students in the beginning psychology 
class in v way which provided the student with his achievement 
in the course. lie felt that many rewards of this system were 
administered by these student proctors. They were able to 
respond in a personal way to the achievements of the students 
at that time when the student was making his response. They 
were able to engage socially in conversation in the subject 
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matter. They 
course a much 
proctors were 
mi gilt have be 
help, 



knew their students personally and made the 
more personal experience. It seemed that these 
able to deliver effectively a program which 
en difficult to correctly implement without their 
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Gallup (A) initiated a programmed study program in his 
Introductory Psychology course at Lafayette College in 1963 
which replaced t lie traditional lecture technique previously 
used. The essential features of his program were (1) the stu- 
dent worked at his own pace, (2) proctors were used to assist 
t lie instructor, (3) unit mastery was prerequisite for progress 
through t he course, and (4) lectures were used for motivational 
purposes. Students were quizzed in a large auditorium whore 
the proctors chocked t lie quizzes when taken. Course informa- 
tion and readiness test performance were posted on a bulletin 
board. Final grades were significantly higher than under the 
lecture format, A questionnaire indicated that students 
worked harder, enjoyed t lie course more, cheated less, and were 
felt to be rccongizcd as individuals, They worried about 
grades tire same as the year before and attitudes toward the 
lecture were unchanged. 

Me Mi chad and Cory instituted a contingency management 
in their General Psychology course at C- W, Post College 
of Long Island University in 1969 (2). Three groups re- 
ceived the lecture format while an experimental group followed 
a Keller format, A significant difference was found between 
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the experimental and each of the control groups. Their ques- 
tions were whether contingency management can be used to teach 
the subject matter of a standard text and whether or not stu- 
dents would learn more. They concluded that students worked 
harder, got better grades and enjoyed the course more. 

HaLott and Svinicki (.&) set up an extensive pro- 
grammed study system in their Introductory Psychology course 
at Western Michigan University, The regular daily quiz 
consisted of two brief construction- type questions. Students 
were given immediate feedback on their per formancc . If the 
student missed either of these questions, he had the opportunity 
of attending one of several make-up quizzes over the reading 
again. The first make-up quiz was given t lie evening of the 
regular quiz and followed a ‘10-ninutc preparation period on the 
day's reading assignment. If tue student did not answer all 
of these questions correctly, or did not attend the make-up 
quizzes, given during the remainder of the day, he came into 
contact with the "Doom's Day Contingency" -- the choice, 
either dropping the course or receiving an ' 1- • fur the entire 
course depending upon the point in the semester at which this 
situation occurred HaLott and Svinicki said that because 
the assignments were sufficiently clear and reasonable, and the 
opportunity to take the quizzes sufficiently frequent, it was 
unnecessary for any conscientious student to cone into contact 
with the ultimate contingency, and few students did, Students 

n 
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in this program worked about 12 nours a week for three hour’s 
credit, and 80 - 00 percent earned a final grade of 'A', while 
less than two percent earned a final grade of 'F'. 

In 1008 Forster (.3) initiated a programed study format 
for his Introductory Psychology course at Georgetown Uni- 
versity, which was sinilar to that enployed by Keller, The 
unique feature is tiic interview used to assess student under- 
standing, After studying a 10-15 page text assignrent the 
student tool, an interview with a "listener" who had already 
read the unit and passed an interview. This listener used a 
tincr and listened to the "speaker" without interruption, 
both students referred to the text or notes as needed. After 
the speaker had finished, the listener comcnted on how the 
speaker covered the topic, nentioning inportant onissions, 
correcting nissunderstandings of concepts or language. If 
both students were satisfied that the interview showed mastery 
of material, they recorded the results on a class chart, and 
the speaker now became the listener and found another student 
needing an interview. Kacli student was required to listen 
once for each tire he spoke. At the end of each three to five 
sections (a chapter), the student took a written quiz to demon- 
strate his mastery. Five or six versions of the quiz were 
available and consisted of essay questions. If a quiz, 
graded oy an assistant, was satisfactory, the student proceeded 




to the next unit of study and interviewing. If his perform- 
ance was not satisfactory, a remedial procedure was discussed 
with the instructor. 

Johnson and Pcnnypackcr (5J at t lie University of 
Florida used "student managers" to conduct tuc testing. 

These were students who themselves had successfully completed 
the course and were enrolled in an Independent >.orh course 
receiving a grade and credit for their efforts as a manager. 

A manager '..’as graded primarily on the performancnc of nis 
students. In the testing setting, the student was given a 
5 tael: of 3 X 5 inch cards containing test items which he read 
aloud and answered while the manager recorded the time. Tnc 
performance measure was the rate of correct responses emitted 
and die rate of incorrect responses emitted. Satisfactory 
performance was defined as no less than 3.5 items per minute 
answered correctly and no more than 1.4 items per minute 
answered incorrectly for each verbal performance on each con 
sccutive unit of subject matter. Cumulative totals ncrc 
plotted on the student’s graph. IXiltiplc form quizzes were 
randomly sampled from an item pool. A student was expected 

to complete two units per week. 

All of the programmed study courses cited have several 

features in common: 

1. Students actively interact with the course material 
rather than function as passive observers. 

2 Progress through the course is contingent upon nns- 

1 
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tcry of the material. 

3. Immediate feedback of their performance is provided 
both in terns of their pcrfocmancc on quizzes and cumulative 
progress to this tine. 

4, The requirement of sustained effort as opposed to 
spasmodic effort frequent in lecture courses with infrequent 
examination . 

5. Provision for remedial work when performance require- 
ment is not attained. 

6, Utilization of students in the capacity of proctors, 
interviewers, or managers. 

The purpose of the present research was (1) to develop 
and refine the technique of management of students' study be- 
havior, (2) to investigate procedures and parameters not a part 
of the early work which might come to light with increased ex- 
perience, and (3) to extend tiic procedure of programmed study 
to academic areas other than the introductory course in psy- 
chology * 

Specifically we arc interested in comparing the effec- 
tiveness of the programmed study procedure with the traditional 
lecture methods in subject natter areas other than the intro- 
ductory psychology course. Within the programmed study format 
it is desired to examine such tilings as the nature and function 
of the proctor; participating students preference and attitude 
data; the necessity for remediation; and contingency manage- 
ment procedure s , 
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la this section a description will be given of (1) a 
three semester comparison of the development of the programmed 
study procedure in t lie General Psychology course and (2) a 
comparison of the traditional lecture format and the programmed 
study format in four courses -- General Psychology, Princi- 
ples of Sociology, Cultural Antrhopoiogy , and Religions of 
the World, 

!D;Asuki;;g i hsiiuc : 

The data gathered in this study uerc obtained by three 
basic typed of instrument -- hourly exams, student performance 
on readiness tests, and attitude measures. 

The Hourly Ilxan : JSajor hourly exams were administered fror 
three to seven tires per course. These exams were included for 
four reasons: (1) they served as a means by which the performance 
of lecture and programmed study groups could be compared; (2) they 
were used to determine the student r s grade for the course; (3) they 
provided minimal pacing for the students in the programmed study 
groups; (4) they imposed a minimum strain on the instructors ii 
charge of the courses -- minirun to the extent that all hourly exams 
were required to be objective exams when such exams were not al /ays 
routinely employed in the courses involved. All these exams were 
composed of from 40 to 70 multiple choice items, generally selected 
ERIC fror the instructor's nrmunl accornnny inr the textbook. Identi- 
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cal tests were given to both lecture and programmed sections in those 
cases where t lie method of instruction was being inves t igated. 

All tests were computer scored and item analysed. Kudcr -Richardson 
Formula 21 reliability coefficients for all hourly exams employed 
were in excess of 0*70, 

The Readiness Tests : This instrument was used to 

assess the student* s mastery of the material lie had studied, 

These tests were short, generally containing from seven to 
ten mult iple-choicc items; but in some courses had several 
essay type items included. Item analysis of these instruments 
was not made; reliability was not a major concern of the in- 
structor who prepared the readiness tests. The instructor 
was more concerned with the face validity of his readiness 
test items. While progress through the programmed study 
course was based on readiness test performance , no statistical 
analysis was attempted on the scores of these tests* 

Measures of student participation in the programmed 
study courses were obtained by computing percent of readiness 
tests taken during the semester. Records were kept of the 
number of test forms needed before the student passed 1 1 1 c 
study unit* His score and the date he took the test were re- 
corded* From this record the percent of work units attempted 
was calculated, 

q All exam and test data were recorded via Tort* A* Punch 
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ansvcr cards rend directly onto a dish file the IpM 11.30 



computer for Inter statistical analyst 



iloasnros of Attitude: An option survey vis given 



to nil prograrred and lecture sections at the completion o r 




by Callup at Lafayette College and a copy is presented in 
Appendix A, Students were ashed to rate the characteris- 
tics described in the question on a continuum running fr or 
"much greater" to "ruich less" as comared to other courses 
which he had talon in his college v:orh to date, All iters 
were rated on a five point scale. Characteristics o r the 
course to be rated were such things ns the arc unt of treasure 
•which the student received by the requi rerents of the course, 
the degree of raster/ which be felt lie was able to obtain 
studying tbc material, tbe degree of achievement which was 
developed by passing tbc readiness tests, the overall enjoyment 
which lie felt was gained by being involved in tbc course, 
t lie degree of rcrori cation which he engaged in in his efforts 
to learn the Material, what influence be felt the instructor 
had on hir in comparison with the influence that other pro- 
fessors had on hir in other courses, what value the course 
scored to put on hir as an individual, to what degree the 
course influenced lis study habits, and finally--as the course 
progressed- -what his general feelings were toward t lie dcsir- 
O ability o r hearing lectures. 
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Osgood's Semantic Differential was used in two courses 
in an attempt to gain some picture as to how t lie students 
viewed the programmed study procedure in particular and edu- 
cational research in general. Each concept to be assessed was 
evaluated on ten scales described by Osgood, shown in Appen- 
dix B. Twelve concepts were thus evaluated and related to the 
course, t lie way it was taught, how one evaluated his study in 
that course, and how one responded to concepts relating to 
educational research . 



Development of the General Psychology Program : 

Eirst Semester : The students receiving the programmed 

study course were required to (1) complete 19 study units, 
(2) meet a criterion performance on at least one of three 



available readiness tests on each study unit, and (3) to take 
five hourly examinations. They also were required to com- 



plete a laboratory sequence which was not an integral part of 



the programmed study course and will not be discussed in this 



report* 




1. Materials : Assigned readings from Kcndlcr's Basic 

Psychology , 2d, cd, Study guides were provided 
with reading assignments and discussion questions 
covering important points in the assignment. 

2* Study Units : Nineteen study units averaging 33 

pages each were used, 

3. Hourly Exams : Five 70-iten nul t ipl c - choice hourly 

exams were given, 



Readiness Tests 
were available 



: Three forms of a readiness test 

for each study unit. These were com- 



90 



4. 
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poscd of ten multiple-choice items similar to those 
composing the hourly exams s 

5* Passing Criterion : A person could proceed to the 

nc x t study unit by either of tv;o ways: (1) passing 

one of the tliree readiness tests with eight out of 
ten items correct; failing form A or B held no 
penality other than preventing the student from 
going on- (2) failing all three forms of the rcadi 
ness test, the student was permitted to novo on to 
tile next study unit, 

6, Credit for Readiness Tests : Three points were 

giverT for eacli readiness tost passed. These points 
were summed and were counted as the sixth hourly 
exam at t lie end of the semesterc 

7, Pacing : Readiness tests were available tliree per- 

iods a week, A student could take as many readiness 
tests as he could pass in a pcriod e However, if lie 
failed, lie was required to rcstudy and return the 
next period. All readiness tests for a given sec- 
tion of the course had to be taken before t lie hourly 
exam on that section was available, This hourly 
exam could be taken during any regular period up to 
a cutoff date (the regularly scheduled date for the 
lecture section to be discussed in the next section) 
Since failure to take an hourly exam resulted in a 
score of zero, no students failed to meet this re- 
quirement , 

8, Proctors : Pour upper division psychology majors 

served as paid proctors. Their function was to 
score readiness tests as the students completed then 
and to discuss the results with each student. In 
practice these proctors generally were able to func- 
tion only in a clerical capacity because of the num- 
ber of students they had to serve. There were 12 
students per proctor, 
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0 , Fornnl Class Meetings : There were no formal 

class meetings afTcr getting the program started at 
the beginning of the scrcstcr, 

Second Semester s During t He second senester the course 
fornat rerained essentially the sane as during the previous 
scrcstcr except as noted in the discussion below* An effort 
was made to increase the students’ motivation by manipulating 
a point system associated with the readiness tests* \n 
attempt vns made to examine the proctor r, ore closely, particti 
larly with respect to who could function ns a proctor. 




1, Materials : essentially the saro as in the previous 

scrcstcr* 

2, Study Units : The number of study units increased 

from 19 to 3 1 by forming smaller units from sore of 
the longer units of tue previous semester, 

3, Hourly Kxrn s : ho eh augo. 

Ileal incss Vests : Two alternate forms, A and i), 

were available, each composed to ten nult irlc •choice 
iters* In lieu of a third form, students failing 
forr B were required to attend a 20-minute reredial 
lecture covering t He unit with which they had diffi- 
culty , 

5, Passing Criterion : bight out of ten iters correct 

on forrs A and 13, Attendance at the remedial lec- 
ture if A and H were failed, 

0, Credit for Readin e ss Tests : In a specific attempt 

to overcome a sense o f fru: t rati on and failure ex- 
pressed by students the previous scrcstcr who failed 
readiness tests, even though the test aided in get- 
ting a better grade on the hourly exam, a point sys- 
tcr was set up as follows; for r A gave five points 
if passed; forr ?> gave four points if passed; and the 
reredial lecture five three points if attended* 

These point values correspond to an A for five 
points, a high D for four points, and C for three 
points. The total aurber of points accumulated by 
ri d* scrcstcr was counted as an h varly exam; accuru- 

99 

t * 
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latcd points during the second half of t lie semester 
were counted as a second hourly exam Thus the 
readiness tests were weighted as two hourly exams, 
or 25 percent of the final grade. 

7. Pacing : A student was required to meet t lie criter- 

ion on either form A, B, or attend remedial lecture 
before being permitted to proceed through the course, 
lie could take the hourly exam when he had completed 
all the readiness tests for the section covered by 
the hourly exaru No date was set except that the 
student must complete all the work by the end of tlic 
semester. As time approached when it would become 
impossible for a student to complete the remaining 
work he was given special couscling by the instructor. 
Two students decided to drop the course, three 
managed to complete on time, 

3. Proctors : Two sets of proctors were used: (1) stu- 

dents who had already taken tlic General Psychology 
course and (2) students who 'were currently enrolled 
in tic course. These latter students had the high- 
est grade point average in tlic course, All proctors 
were paid by the hour. The proctor- student ratio 
was one proctor per eight students. 

9. formal Class l!cctinns : None. 

During this semester a change in the class environment was 
made* A large lecture hall was used as a study hall where 
students could come and read prior to taking a readiness 
test, Initially attendance was checked and students were lead 
to believe that they were expected to be there at least at tlic 
beginning of tlic three periods scheduled for tlic course each 
week, While tlic number of students using this study hall 
decreased as tlic semester progressed, it was used by a con- 
siderable number at all times, A second room was set aside in 
which readiness materials were handed out and collected and 
where the students took the readiness tests. A third room was 
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reserved for the proctors. After reporting 
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rcadincss test, the student went to his proctor to discuss his 
performance , After going over his test, and discussing any 
problems or concepts he might wish, the proctor plotted the 
student's performance on his cumulative record and showed him 
his progress relative to his rate of test taking and the end 
of the semester, A copy of the recording sheet used to keep 
the cumulative record is shown in Appendix C e A record of 
his performance on the hourly exams was also kept on this form, 
Thus a student knew how he was performing at all tires, 

Remedial lectures were given by a faculty member in a fourth 
room, Hourly exans were taken under supervision in the same 
room that the readiness tests were given. 

In all of the courses included in this study the sarc text- 
books were used by both lecture and programmed study sections, 
Likewise t lie sane objective hourly exams were given to both 
groups, T-scorcs were employed wherein the parameters were 
from t lie lecture section. Tills was done in order that the 
students in the lecture sections would not be put at a disad- 
vantage gradevise by participating in this project, Final 
grade distributions arc reported on t lie basis of T-scores. 

All statistical analyses are done on raw exam scores. 

Third Semester : In the third semester a special effort 

was made to devise a systen in which the student would develop 
better vcrual skills in discussing t he course materials, An 
attempt was also made to engender interest and enthusiasm for 
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participation in the course, 

1* Mater ials : In addition to study guides as previous- 

ly used, the discussion questions covering the text 
assignment were to serve as items on the readiness 
tests. A new text was used: Krctch, Crutchfield 

and Livson, hlements of Psychology , 2de cd s 

2, Study Units : No change e 

3. Hourly Exams ; No change, 

4, Readiness Tests : Two forms composed of five 

multiple- choice items and two essay items randomly 
selected by the proctor from the study guide questions. 
Students answered these two questions orally e In 

lieu of a third form of the readiness test, the stu- 
dent was given a rather lengthy fill- in- thc-blank 
exercise taken from the student workbook accompanying 
the text, but not otherwise used, The student took 
this exercise and completed it, using his textbook if 
nescssary, and checked his work with his proctor be- 
fore leaving, 

5. Passing Criterion ; The student was required to 
get four out oITTive mult iplc- choice items correct, 
and both essay questions correct in order to continue 
with t lie next study unit: 

6 ♦ Credit for Readiness Tests ; The point system was 
essentially the sane as in the previous semester. 

7, Pacing : No change. 

8, Proctors; The proctoring staff was composed of 
students chosen on the basis of their grade point 
average and were contacted following the preregis- 
tration period proceeding t he semester they were to 
proctor. Instead of being paid, they were registered 
in a one-hour independent study course in the psychol- 
ogy department- In addition to proctoring duties, one 
period a week was spent with the instructor. During 
this period not only did they discuss course material 
but also how their students were performing, ways of 
helping then to do better, and ways to engender 
greater interest and enthusiasm for the course, Each 
proctor was assigned six students Test data on 
these proctors is not included in the results, 

9, Formal Class Meetings : None, 
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A different textbook was employed this third semester 
which was longer. The number of study units was increased, 
but tie average length of assignment was decreased over the 
first semester. Comparing the third semester with the first 
there was a 79 percent increase in the number of study units 
to be completed; a 28 percent increase in t lie total amount of 
material to be studied; and a 37 percent decrease in the 
number of pages per unit. 

Comparison Between Lecture and Programmed Study Procedures : 

The first and third semesters of General Psychology 
course previously described were run in an experimental-control 
group design. Only the experimental groups (programmed study) 
were described. It was desired to compare the relative per- 
formance resulting from these two course formats. Since most 
of the investigations have previously employed the General 
Psychology course, it was decided to incorporate courses in 
other departments. Selection was a function of numerous 
considerations. Courses with large classes v/crc desired over 
courses with small ones. The interest and cooperation of the 
instructor was a prerequisite; tins eliminated some of the 
courses one night have liked to vise. Those instructors who 
participated constituted popular ones in student’s eyes; teach- 
ers who were willing to be innovative in their teaching, Sore 
have a rather long history of educational innovation. 

In addition to two different General Psychology courses, 
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courses in Educational Psychology, Cultural Ant hropo logy , 
Principles of Sociology, and Religions of the World were inves- 
tigated* In setting up the study units, the instructors were 
counseled, but ultimately they made the final decisions as to 
materials to be used, amount of material to be covered, size of 
study unit, type of readiness test, and pacing features. It 
was desired that this research result in a minimum of "bugging" 
of the non-psychology courses , 

Other courses were originally included in this part of the 
study, but were subsequently dropped because of a breakdown of 
experimental control of one type or another which was discov- 
ered too late to be remedied. In one course in which differ- 
ent types of activitcs of the proctors were under investigation, 
half-way through the semester it was discovered that some of 
the proctors had simply changed their method of proctoring in 
order to serve their students more effectively, thereby 
invalidating the results. In an attempt to gather student 
choice data, lecture and programmed study sections were 
scheduled in the sane courses. However, it was discovered to 
the horror of the authors that the instructors of these courses 
had scheduled the two sections at different hours. In question- 
ing tli e students in these courses it became apparent that their 
choice of sections was influenced more by their schedule that 
semester. All was not in vain; t lie authors learned a good deal 
about non-laboratory research while increasing their pessimism 
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about permitting experimental control to get out of their own 
hands , 

Kadi course was divided into two sections, one of which v/as 
taught by the lecture method and the other taught by the pro- 
grammed study procedure. In all eases the students enrolled 
in a particular course ./ere ranked according to their cumula- 
tive grade point average and assigned by a randomized block 
design to lecture or programmed sections, 

In the material t hat follows a description of the non- 
psychology courses used in this study v/ill be given, using the 
outline previously employed in describing the fcncral Psychol- 
ogy programs, Outcome of these various procedures will be 
discussed under "Results". 

Development of t lie Cultural Anthropology Program: 
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1, Mate ri a Is : Study guides gave reading assignments 

only, Assigned readings were from llocbcl , Anthro - 
pology. The Study of Man , 3d, c d t 

2, Study Units : Twenty-nine study units averaging 

13 pages of text material, 

3* Hourly Kxar.s : All students in both programmed stu- 

dy and lecture sections were required to take the six 
hourly tests at the same hour on the date scheduled 
at the beginning of the course, If a student in the 
programmed section finished t he readiness tests early 
lie was required to wait until the announced time to 
take the hourly exam, 

4, Readiness Tests : Two forms of readiness tests 

were aval 1 able for each of the 2J study units, These 
were composed of eight mult iplc-choicc test iters. 
Five periods per week were set aside for taking 
readiness tests . 

5. 



Passing Cr i ter ion : 



Six out of eight items (73 per- 

PS 
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cent) were required to pass readiness tests, A stu- 
dent was not required to take and pass all the readi- 
ness tests, 

6. Credit for Readiness Tests ; If students tool; and 
passed all the readiness tests in a given section of 
the course, he received a bonus of four points added 
to his hourly exam score, 

7, Pacing: dourly exams weref available during any 

readincs s test period, Uork had to be completed by 
the end of the semester- 

•27 

3, Proctors : One proctor was available for ^9* students 

and served in a clerical role of passing out and col- 
lecting test materials, scoring tests as the student 
finished. The proctor was an upper division sociology 
major who would answer questions if asked, Because 
of having five periods a wed for rcadinscs testing, 
the number of students taking readiness tests at a 
given period was thereby substantially reduced, 

0, Formal Class Meet inns : Thirteen formal meetings 

of the entire programmed study group were held dur- 
ing the semester, roughly one a week. 

Second Semester : In the second semester an effort was 

made to increase student participation by reducing the passing 
criterion while requiring that all readiness tests be taken, 
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1, Materials : ho change. 



2. Study Units ; ho change, 

3. Hourly kxnns : ho change, 

4. head incs s Tests : Two forms of each readiness 

test containing both* multiple- choice and essay iters. 



Passing Criterion : The passing criterion was 

lowered Trcvn~ the previous six out of eight to five out 
of eight (63 percent). However, students were re- 
quired to take nil readiness tests. One could pro- 
ceed onto the next study unit either (1) when he had 
onssed one o r the two forms of the readiness test, 
or (2) when he hml passed an oral quiz giver, by the 
instructor- All units were required to be passed in 
order to get a grade above "P u . 
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Credit for r.cruliness Test : 
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7, Pacing: No change * 

8, Proctors r Five proctors we re provided, giving a 
ratio of eight students per proctor. 

9, Formal Class Meetings : Fight times during the 

semseter, generally after an hourly exam, the pro- 
grammed study group met together with the instructor. 

Development of the Educational Psychology Program : 

1. Materials : No study guides were used. Students 

received an assignment sheet for the semester. Texts 
used were Gibson, Iiducat ional Psychology (A Pro- 
grammed Text) ; Holland and Skinner. Analysis of 
Behavior (A Programmed Text) ; and Skinner. The 
Technology of Teaching . 

2. Study Units : Twenty-five study units averaging 

15 pages each were used. 

3. Hourly Fxams r Three 50-item multiple-choice exams 
were givenT 

4. Readincs s Tes ts : Two forms of each readiness test 

were available 7 These contained eight multiple- 
choice and short answer items, and were available 
during four periods each week. 

5. Pass ing Criterion : A score of 75 percent was re- 

quired for passing. Students were not required to 
take and pass all readiness tests in order to progres 
through the course, 

0, Credit for Readiness Tests; Same as for Anthro- 
pology. 

7, Pacing : Hourly exams were available during any rcadi 

ness test period. Work had to be completed by the 
end of the semester, 

8, Proctors : Sane as for first semester Anthropology 

9, Formal Class Meetings : None , 

Development of the Principles of Sociology Program : 

Materials : No study guides were used. Students 

received assignment sheets at the beginning of t lie 
semester. The text was Broom and Selznck, 
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Sociology , 5th ed c 

2, Study Units : Twenty-five study units were used, 

averaging IS pages per unit. 

3, Hourly Exams : Three hourly exams were given. 

These were part objective and part essay, Only 
the objective items were used in the data analysis. 

4, Readiness Tests ; Two forms of readiness tests 
were available for each of the 25 study units. 

These tests had eight multiple-choice items, Four 
testing periods a week were provided, 

5, Passing Criterion : Six out of eight items (75 

percent) were required to pass, A student was en- 
couraged to take the readiness tests, but was not 
required to do so to continue progress through the 
course . 

6, Credit for Readiness Tests : Uo extra credit was 

given for taking tire readiness tests, 

7, Pacing: Same as for first semester Anthropology , 

8, Proctors ; Same as for first semester Anthropology . 

9, Formal Class Meetings : One meeting of the entire 

programmed study group per week; 15 meetings for the 
semester. 

Development of the Religions of the World Program : 

1, Mate r in Is : Study guides were provided which con- 

tained the study assignment and discussion questions 
to point up important aspects of t lie assignment. 
Reading assignments were in lioss, Han 1 s Pel igion , 
4th, ed, 

2, Study Units : Thirteen study units averaging 70 

pages each- 

3, Hourly Exams ; Seven hourly exams were given, 
roughly" on every two study units, These were 
70-itcn multiple-choice exams. 

4* Readiness Tests : Three forms of each readiness 

test were available, each ten items long. Four 
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tcsting periods per week v;erc provided: 

5, Passing Criterion : Eight correct out of ten items 

(80 percent) were required for passing. Students 
were not required to take and pass all of the rcadi 
ness tests in order to progress through the course. 

7. Credit for Readiness Tests : Same as for Anthro- 

pology e 

8. Proctors : Sane as for first semester Anthropo logy 

0t Formal Class Meetings : Seven class meetings, 

one after each hourly exam. 

Special Investigations : 

General Psychology t 4th Semester: A section of 

students in General Psychology was used in a 2 X 2 factor- 
ial study to examine the parameters of study unit length and 
active-passive proctoring, The course arrangement s were 
essentially the sane as described under General Psychology - 
first semester except as noted below. Students were ranked 
on grade point average and assigned by a randomized block dc 
sign to four groups. 

Groups 1 and 2 used the programmed study procedure with 
39 study units, Groups 3 and 4 likewise employed the pro- 
grammed study procedure but with only 13 study units, each 
three times as long as used by groups 1 and 2: The sane 
materials were used by all groups. The "long" study unit 
group took the sane readiness test questions as the "short" 
group, except three of the short, ten-question tests were 
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taken at a tine. 
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Proctor ing was done by upper division students who had 
completed tie course previously and who were paid for their 
services* The proctors in Groups 1 and 3 were instructed 
simply to handle the clerical details of passing out readi- 
ness tests and checking answers. They were to refer any 
questions asked to the instructor. The proctors for Groups 
2 and 4 were instructed to go over each readiness test with 
t lie student as soon as lie had finished taking it, having him 
explain his answers and giving supplementary or remedial ex- 
planations when neededt By mid-semester it became apparent 
that these proctors were not able to carry out their instruc- 
tions; some of them in groups 1 and 3 were performing more 
like those in the other two groups, and vice versa. There- 
fore, no attempt at analysis of this variable was attempted. 

hducat ional Psychology , second semester: A class of 

64* students in Educational Psychology was used to examine 
study unit size, One group used short 15-pagc reading as- 
signments* and the other 45-page reading assighments. Students 
were assigned to the two groups on the basis of a randomized 
block design utilizing grade point average* Other charac- 
teristics of tire program were like those previously described 
under the first semester of Educational Psychology^ 
er i j tie -e of the pr o gram we r e — li k e pr e vi o ndy d escrib e d 

u nd e r E duc at i o n al ^cychol ogy , fir s t sem este r 
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Social Psychology: One section of the Social Psych- 
ology was used to cxaninc the usefulness of snail group dis- 
cussio?is within a programmed study format. 

The group discussion was seen as the way of adding addi- 
tional rewards to the program of which it was a part. The 
instructor using discussion groups was running thcr with four 
basic ideals in nind through which they could make the program 
a r ore rewarding experience, First, lie considered the dis- 
cussion group as a novel experience within the program format, 
The number of discussion groups were kept at ten per semester 
so as not to lose this aspect of novelty. Since the general 
routine within a programmed class is to core to the proctor , 
take a quiz, tall: to the proctor briefly, and leave, it was 
felt that some change in this overall routine would be an ex- 
perience which would serve to increase the reward aspect of the 
total program. The second point was that in these groups the 
student could have a closer interaction with the professor- 
Prom the report of many students, they had indicated tint to 
have access to the professor, whether it was used ns a point 
of contact or not was a desirable condition, he have not 
checked to see if this is true for the majority of students 
but for those students whom we have interviewed professor con- 
tact would appear to be a rewarding experience, 

Pi 



The third aspect 



of the group discussion was that this was a tine in which the 
concepts covered in the text would be rehearsed. The discus- 
sion engaged in by the group would involve the use of t lie 
terns and concepts which had been part of the reading assignment. 
In the group session, discretions of social settings would 
be made using the terns and concepts' employed by professionals 
in the field. This was done to add a degree of utility to 
t lie reading material, The fourth emphasis of the group dis- 
cussion was to encourage the report of experiences which the 
student had which were in line with t lie subject natter of that 
particular course. The reward was the enjoyment gained in 
relating their experiences to others. To relate one's experi- 
ence appears to be a type of communication which can be done 
more easily than many other kinds of verbal performance. The 
above four points arc the basic conceptualization of the re- 
ward aspects of the group discussion experience as used in our 
program, The hoped for effect was that the student would be- 
come more positive in his attitude toward the overall program 
as well as increase their frequency of participation in the 
program, 

Several types of activities were provided for the groups 
to facilitate their achievement of the desired goals. In- 
cluded were variations of activity from week to v: ec>; in an 
effort to keep the experience of the group meeting somewhat 
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novel, Several of the meetings revolved around interesting 
articles which were pertinent to the subject matter field. 
There were other meetings in which role play was used as a 
procedure by which t lie student could more easily use the terns 
and concepts employed in the text he was studying, Other 
sessions were devoted to relating one's experience in relation 
to the particular subject matter being studied, These were 
the three basic types of experience provided for in the group 
sessions, Many sessions combined two of these activities, 
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RLSULTS 



Performance on Hourly bxams : 

Table 1 presents a brief descriptive summary of each 
course discussed in this report. liach time one of these 
courses is referred to in the section on Results it will be 
followed by 3 parenthetical reference to the line in Table 1 
describing that course. This lias been done so as to mini- 
mize possible confusion among the thirteen courses. 

Table 2 shows the mean exam scores for the semester 
for tie lecture and programmed study sections of the seven 
courses involved in this project. The two General Psychol- 
ogy courses, the Lducational Psychology course, and the 
second semester of Cultural Anthropology showed a statisti- 
cally significant difference with the programmed study sec- 
tions having the higher mean score, 'flic remaining courses 
failed to show any significant differences between lecture and 
programmed study procedures. Table 3 siiows the number of 
hourly exams on which t lie programmed study section scored 
higher than the lecture section for these seven courses. 

In the General Psychology and Lducational Psychology 

.5 

(lines 1,3, -+5 courses there was a significant difference 
between lecture and programmed study groups on all hourly 
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exams given during the semester as shown in Table 2. In the 
second semester of Cultural Anthropology (Table 1, line 9) 
the difference between lecture and programmed study groups 
became significant only at the end of the semester, and shows 
up in pooled scores. 

Table 4 shows the percent distribution of final hourly 
exam averages for the three semesters of General Psychology 
(lines 1, 2, 3). The Kolmogorov-Smi rnov two-sample, one- 
tailed test indicates a significant Chi-Square for differences 
between lecture and programmed study sections, both for the 
first aid third semesters. 

The difference in the distribution of final averages 
between t lie programmed study group of the first and third 
semesters is not significant. 

Figure 1 shows the major exam-taking rate for the long 
and short study unit groups in the Educational Psychology 
class (Table 1, line 6), This is a cumulative plot of t lie 
mean number of exams taken on each of the 16 days pro- 
ceeding the required date for the hourly exam. These data 
are based on the means of three such pre-exam periods . 

The "short" study unit group having three times as many 
quizzes still arrived at the hourly exam on the average more 
quickly than did the group which had the longer assignments 
and fewer work units. 
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Mean Hourly Exam Score Distribution for General Psychology 
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Significant at .05 Level of Confidence 
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Flgure 1. emulative moan work rate per day as a 
function of assignment length in Educational Psychology 
(line 6) 
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No interpretation will be attempted of the apparent FI 
scalloping suggested in the cumulative record for the short 
study unit group* 

Tables 5 and 6 give t lie results of an analysis of 
variance of mean hourly exam scores for the "short" and "long" 
study unit groups in the Educational and General Psychology 
classes respectively (lines 4, 6). In both cases assignment 
length is significant beyond the .05 level of confidence, one- 
tail test. No significant interactions were observed. 

Figures 2 and 3 show a plot of the mean hourly exam 
scores for the semester as a function of grade point level. 

The short study unit groups scored higher on the hourly 
exams at all grade point levels. 

Table 7 shows the results of an analysis of the "long" 
and "short" study unit groups utilizing percent of readiness 
tests attempted by students in these two groups, and the 
percent actually passed of those attempted. No statistically 
significant differences in the means were found. An opposite 
effect of study length means is observed between the General 
and Educational Psychology courses which will be discussed 
later in terms of course difficulty. 

Participation and Performance: 

As an indication of participation in the programed study 
sections, percent of students taking each of the available 
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TABLE 5 

Analysis of Variance of Student 
Performance in Educational Psychology 



Source 


OF 


; F 


“1 


Assignment Length 


1 


Kj 0 • — 0 ** • *Y 2 * 


i 

* 


6PA Level 


3 




i 


ilethod X Level 


3 


16.35 


i 

j 


Error 


56 


19.46 




Total 


63 




t 

j 


,* PC. 05 


TABLE 6 






Analysis of Variance 


of Student 




Performance 


in General 


Psychology 




Source 


OF 


rs F 


i 

t 


Assignment Length 


1 


2407.94 4.30* 


{ 

! 

( 


GPA Level 


2 


020.54 


i 


Method X Level 


2 


59.63 


t 

f 


Error 


24 


585.31 


i 


Total 


29 




i 

i 



* P <.05 
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Figure 2. Mean hourly exam performance as a function of 
assignment length and GPA level in Educational Psychology 
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Figure 3. 1-iean hourly exam performance as a function 
of assignment length and GPA level in General Psychology. 
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rcadiness tests was computed. This data revealed changes 
in participation across the semester. Tabic 8 presents 
the percent participation in the programmed sections by indi- 
vidual study units in eight courses having programmed study 
sections. Cell entries are the percent of students in that 
section completing the unit; the final column is a mean per- 
cent across all study units. The courses arc ordered from 
that with the highest participation to the lowest* Table 8 
also, incidentally indicates the number of study units 
in each course. The third semester General Psychology 
(line 3) and second semester Cultural Anthropology {line 9) 
have the highest rate of participation. 

Table 9 presents percent of readiness test passed of 
those taken in the programmed study sections by individual 
work units. Those courses whose cont ingencics required passing 
before proceeding on have been eliminated from this table. 

The last column contains mean percent across all study units 
and can be interpreted as an index of course difficulty, 

A Pearson Product Moment correlation between percent 
passed and percent attempted across the five courses shown 
in Table 9 (N a 443) yields r a 0.410. At the ,01 level 
of confidence a significant r a 0.128, 

Figure 4 shows the relation between grade point average 
and percent participation, based on those sections with mini- 
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Percent Participation in the Programmed Study Sections 
by Individual Study Units in Seven Courses 
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Figure 4. Percent participation as a function of GPA in 
sections with minimal requirement for taking readiness 
tests. 
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mal requirement for taking readiness tests, As t lie grade point 
average involved is higher, the percent of study units taken 
is higher, A Pearson Product Moment correlation between 
t lie percent of the study units passed and the mean hourly exam 
score wis r = 0,167, As the ,01 level of confidence a 
significant r 3 0,128, 

Figure 5 shows the percent participation as a function 
of study unit length for five courses which have been ordered 
from the one with the shortest study units to the one with 
the longest study units, Since these courses differ in 
other ways also, care must be taken in interpreting this 
f i gure , 

Use of Alternate Readiness Test Forms : 

The three semester sequence of t ii c General Psychology 
course (lines 1, 2, 3) gives information about the utilization 
of t iie various forms of the available readiness test, Three 
forms were available for each of the study units, Table 10 
presents t lie mean percentages with which forms A, B, and C 
were taken and passed, A fourth category, "not passed", con- 
tains failure on form C and non-at tempts , The first semester 
of tins course did not involve a rigorous requirement that all 
the readiness tests be taken; the second and third semesters 
did have this requirement for progress through the course. 

In all three cases a majority of the students passed on the 
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Relative Unit Size 



Figure 5. Graph of the percent participation as a function of 
study unit length in five programmed study courses in which 
the request to take readiness tests was not manditory for 
taking hourly exams. 
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TABLE 10 

Utilization of Alternate Forms of Readiness 
Tests in Three Semesters of General Psychology 
(I'ean percent passing) 



Semester 


N 1 


j Readiness les 


c ronn 


Not Passing 


A 


B | 


1 c 


1 


60 


Wi 


8 


5 


30 


2 


60 


81 


15 


4 


0 


3 


35 


78 


14 


8 


0 




r*n 
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first attempt. As passing the quizzes became contingent for 
progressing through the program, the percent passing form A 
increased, A relatively few students required the third form, 

Performance and Grade Point Averag e : 

Figures 2 and 3, previously cited, show a direct rela- 
tionship between grade point average and hourly exam per- 
formance, The relationship between these variables is also 
shown in the results of the Cultural Anthropology courses. 

In the first semester (line 3) there was no significant 
differences between the exam performance of the lecture and 
programmed study groups, In the second semester significant 
differences were observed, This second semester of Cultural 
An tli ropo logy (line 9) is viewed as representing the best 
arrangement obtained in non-psychology courses, and for that 
reason will be discussed in more detail. 

Figure 6 compares t lie mean hourly exam scores for the 
lecture and programmed study groups in each third of the grade 
point range for the first semester of Cultural Ant tropology . 
Table 11 presents results of an analysis of variance of these 
data, No systematic difference between the two groups is 
indicated although overall there is a significant increase 
in exam performance as a function of grade point average. 

Figure 7 gives the sane comparison for t lie second semes- 
ter of the Cultural Anthropology course. Table 12 presents 
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Figure 6. Comparison of Lecture and Programed Study- 
Sections of Cultural Anthropology, First Semester (line B) 
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TABLE 11 

Analysis of Variance of Lecture Vs 
Programmed Study Method 
Cultural Anthropology, First Semester 



Source 


OF 


MS 


E 


Method 


1 


95.09 


0.11 


GPA Level 


2 


6230.19 




Method X GPA 


2 


334.85 




Error 


48 


897.22 




Total 


53 
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Analysis of Variance 


of Lecture Vs 






Programmed Study Method 




Cultural Anthropology, Second Semester 




Source 


OF 


MS 


F 

.. 


Method 


1 


3400.22 


4.76* 


GPA Level 


2 


1053.01 




Method X GPA 


2 


713.42 




Error 


48 






Total 


53 
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Figure 7, Comparison of Lecture and Programmed Study 
Sections of Cultural Anthropology, Second Semester 
(line 9) 
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results of analysis of variance of these data and shows a 
significant difference between methods with the programmed 
study group scoring higher than the lecture at all three grade 
point ranges . 

Figure 8 makes the same comparison for the third semes- 
ter of General Psychology, which has achieved the best 
arrangement of the programmed study format. As with the 
second semester of Anthropology , these data show the program- 
med study group scoring higher at the two lower grade point 
average intervals than the lecture, but not different at the 
highest. Table 13 presents results of the analysis of vari- 
ance of these data* 

In the Sociology course (line 11) students were given 
a choice between a lecture or programmed study sections. 

There was a misunders tanding of the range of application of 
the term ’’choice" and it was later discovered that those 
students choosing the programmed study section were given a 
further option of choosing whether or not to take t lie readi- 
ness tests. After this was discovered it was agreed that 
all students in the programmed study section would be required 
to take all of the readiness v .ests of the last third of the 
course in order to be allowed to take the hourly exam on that 
material. As a result we have data on t lie same set of 
students showing optional taking of readiness tests and the 
resulting hourly exam scores, and required taking of readiness 
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Grade Point Average 

Figure 8. Comparison of Lecture and Programmed Study 
Sections of General Psychology, Third Semester {line 3) 
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TABLE 13 



Analysis of Variance of Lecture Vs 
Programmed Study Method 
General Psychology, Third Semester 



Source 


OF 


MS 


F 


Method 


i 


2719.53 


5.72* 


GPA Level 


2 


442.56 




Method X GPA 


2 


1189.76 




Error 


66 


475.15 




Total 


71 







* PC05 
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tests and the resulting hourly exam scores. Figure 9 shows 
the number of readiness tests taken by the members of the 
programmed study section which were associated with each of 
the three hourly exams. Also the scores on these three hourly 
exams are plotted. In the last third of the course the rate of 
readiness test taking greatly increased, and there was a 
corresponding increase in the mean hourly exam performance. 

Table 19 gives the results of the Analysis of Variance for 
repeated measures for this data. A significant difference 
between exams was found. 

Long Term Program liffecti vcncss : 

The first General Psychology class to be taught by the 
programmed study technique was retested fifteen months after 
they had completed the course. About one half of the stu- 
dents in the class responded to a request for their reevaluation. 
Of those responding 554 were from the programmed group and 
454 were from the lecture group. 

Table 14 shows the mean group performance on the fifteen 
months retest as well as the results of the pro- and post- 
standarized criteria test given these students at the beginning 
and tire end of tire course. 

The criteria test results indicate that both groups 
increased in their ability to respond correctly to questions 
concerning General Psychology. The programmed group had 

(•: 





Hourly Exam Humber 

Figure 9. Moan performance on three hourly exams in 
Principles of Sociology compared with mean nunber of 
study units taken prior to the exam, (line 11) 








Mean Number of Study Units Taken 



Analysis of Variance of Performance 
on Three Hourly Exams in 
Principles of Sociology 



Source 


DF 


MS 


F 


Between Ss 


36 


232.8 




Within Ss 


75 


66.2 




Treatment 


2 


433.7 


7.7* 


Residual 


73 


56.0 




Total 


111 


120.7 





* P < .01 
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TABLE 14 



Analysis of Long Tern Program 
Effectiveness in General Psychology 





Precriteria 

Text 


Postcriteria 

Test 


Exam 

Average 


15 Month 
Re -exam 


Lecture 

Group 

Mean 


57 


98 


127 


95 


Programmed 

Group 

Mean 


57 


110 


i 

138 


i 

99 


t value 


.33 


1.06 


2.79* 


.45 



* P <.01 







a slightly greater increase as indicated by the group mean, 
but this increase was not large enough to consider it a reli- 
able difference in the group's performance brought about by 
the programmed study method. 

The programmed study technique did have a significant 
effect on the students' ability to score while in the course. 
The fifteen month time lapse saw both groups losing about 
the same amount of what they nad learned (25* lecture group 
versus 28% programmed group), The performance of the two 
groups after being away from the course material for fifteen 
months could be described as similar= The treatment had no 
measurable effect on their present ability to answer questions 
concerning General Psychology after this period of time. 

Subject Matter Knowledge of Proctors : 

Table 15 presents data comparing student performance of 
students who had proctors who had already taken the course 
with those who had proctors who themselves were currently 
enrolled in the course, These two types of proctors will be 
referred to as 'out-of-class 1 proctors and ’in-class- 
proctors 1 respectively, 

No significant differences were found between mean 
scores on t lie hourly tests between students having the two 
kinds of proctors, Likewise, no significant differences were 
found in attitude toward the course and toward proctors, 
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TABLE 15 

Comparison of "In-Class" and "Out-of-Class* Proctors 
on Mean Hourly Exam Scores and Item 9 of Attitude 
Scale (enjoyed) for General Psychology (4) 



Measure 


In-Class 


Out-of-Class 


Mean 


so 


Mean 


SD 


Mean Hourly Exam 
Score 


126.39 


24.49 


136.37 


25*35 


Mean Scale Value 
for Item 9 on 
Attitude Scale 


3.33 

c oo 

o ; 


0.65 


?SS 

-6r$5 


0.92 




f'S 
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Contingent Small Group Discussion : 

Table 16 shows mean performance and participation data 
from two sections of Social Psychology using small discussion 
groups receiving credit for attendance at discussion sessions, 
and small discussion groups receiving credit for performance 
during tire discussion sessions. Contingent performance 
showed higher mean hourly exam performances and higher 
mean percent participation in readiness tests than 
the contingent attendance groups. 

Figure 11 shows the response of the students in the three 
semesters of General Psychology on eight items of the opin- 
ion survey. The students in these three semesters tend to 
agree that the programmed course in General Psychology is a 
bit more work th a they are generally used to in general edu- 
cation courses, They also agree that their unders tanding 
of the basic concepts presented in the course arc better 
understood when presented in the programmed study format. The 
second and third semesters show increased pressure, mastery 
and enjoyment over the initial semester. 

Student Op inion Related to the Choice of Course Section : 
Attitudes observed with this pre-course survey noted 
some significant differences in point of view of the 
students in the two sections of the Sociology and Religions of 
the World courses. The attitudes held by those students who 
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Mean Performance and Participation Scores 
Two Groups in Social Psychology 



Measure 


Small Groups with 
Credit for Attendance 


Small Groups with 
Credit for Performance 


Mean Hourly Exam 
Score 


54.2 


59.4 


Mean Percent 
Participation 


692 


812 
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Amount of work required by program 
Degree of pressure experienced 
Level of mastery achieved 
Understanding of basic concents 
Enjoyment of program 
Influence of instructor 
Efficiency of study habits 
Desire to attend lectures 
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4—d 2d Semester 
o — © 3d Semester 



Figure 11. Summary of Student Attitudes Toward 
Programmed Study Courses in Three Semesters of 
General Psychology 
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registered for the programmed section of both courses were 
quite similar. This was also true of those who registered for 
the lecture course. In both courses the most striking dif- 
ference in attitude was seen on the item concerning how much th 
student would enjoy programmed courses as compared to a lecture 
type course. In both courses the lecture group felt that 
they would enjoy it somewhat less than a lecture course, where- 
as the students who had chosen the programmed section felt 
they would enjoy this method a bit more than a lecture course, 
Other differences which arc similar for the programmed section 
versus the lecture section are in regards to their feeling 
about their understanding of basic concepts in the programmed 
course arid the level of mastery of tnc specific units of mater- 
ial in the programmed course. In botii cases the ones choosing 
the programmed course felt that their unders tand j ng and mastery 
would be above average as compared to a lectur^ course, where- 
as the lecture section student felt that the programmed course 
would produce somewhat below average understanding and 
mastery. On the various attitude dimensions observed those 
who chose the programmed section viewed it as a positive ex- 
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pericncc and those who chose the lecture section viewed the 
programmed section in a less favorable way. These results 
arc shown in Picture 12 and 13 

Table 17 shows that only about half the students checked 
the method of instruction as the explanation for their choice 
of section. In the Sociology class, which was made up of 
72 students, 38 of these students Jiosc the programmed section 
and 34 chose the lecture section- Of these 7 i students, 22 
percent of them said they chose tne programmed section because 
it was t ire preferred technique lor then, whereas 17 percent 
said they chose the IcctuTC section because it was the method 
of instruction which they preferred- Sixty* one percent of 
tlic students in the Sociology class made their decision as 
to the section they wanted on some variable other than the 
teaching technique- The same general trend was observed in 
tlie Religions of the World course Of the 0 1 students in 
tli i s class, 2 8 chose the programmed section and 33 chose t lie 
lecture section Thirty two percent of the students said 
they chose the programmed study technique because of its being 
t lie preferred instructional technique for then whereas 23 
percent chose tire led me technique for this reason In t lie 
course 45 percent of the students chose their particular sec- 
tion for some reason other than that of instructional tech- 
nique 

In figure 3J the percent of successful participation 



- 69 - 



Much About Much 

Greater Same Less 

1 — J 



Difficulty of Programed Course 

**Leval of Mastery in Programmed Course 

Final Grade made possible by a 
Programed Course 

Degree of Pressure Experienced in 
Programmed Course 

Amount of Work Required by a 
Programmed Course 

^Understanding of Basic Concepts In 
Programmed Course 

**EnJoyment of Programmed Course 

Influence of Instructor In 
Programed Course 




figure 12. Pre-course Summary of Student Attitudes Toward 
Programmed Study Course, Principles of Sociology 
Programmed Section, o — o Lecture Section, 

**lndlcates a significant t at P<,05. 
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Enjoyment of Programmed Course 
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Influence of Instructor In 
Programmed Course 
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Figure 13, Pre-course Summary of Student Attitudes Toward 
Programmed Study Course, Religions of the World, 

Programmed Section, o--olecture Section, 

^Indicates a significant t at P<*05. 
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TABLE 17 



Analysis of Student Choice In two 
Volunteer Programmed Study Courses 



i 

j 

i 

j 

j 

j 

j 

j 

j 

i 



Reason for Choice 


Sociology 


Re 1 1 gi on 


Preferred Programmed Study 
*» Procedure 


22 % 


32 


Preferred Lecture 


17 


23 


Preferred the Time of Day 


22 


1 1 


Reason not listed 


39 


34 


Number of Students In Cfass 


72 


61 




ERIC 




Figure 14. Student attitude on a five point scale is recorded for 
five courses ranked according to mean percent successful participation for 
the class for six attitude categories. A^Educati ona I Psychol., B=Sociology 
ORcligions of the V/orld, 0= General Psychol., E=Cuitural Anthropology 
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is conpared with regards to six attutudc dimensions. These 
arc the degree of pressure which the course seems to exert on 
the student, the degree of mastery which he feels was gained 
through the course, his overall enjoyment of the course, his 
feeling concerning the length of the assignment given in the 
course, and his feelings concerning development of study 
habits within the course. 

The comparison in Figure IS shows six attitude dimen- 
sions compared for two courses both of which were offered early 
in the program and again in the final semester. Student 
attitude is very similar between the earlier and later pro- 
grams, In General Psychology there was a slight drop in 
the feeling of pressure in the course as well as in the amount 
of enjoyment* In Anthropology there was an increase in the 
feeling that they were developing better study habits and a 
decrease in concern over the length of the study assignment. 

In figure 1G the results of the Semantic Differential 
Test arc shown for Sociology and Anthropology . The concept 
of "fun" on these scales for these two classes fell somewhere 
between eight and nine and the concept of "evil” fell somewhere 
between two and three on a ten point scale, In the compar- 
ison of the response of the classes it was observed that there 
were five of the twelve scales in the Cultural Anthropology 
class in which the lecture and the programmed groups respond- 
ed more favorably to the concept of Anthropology and the 
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Figure 15- Student attitude on a five point scale as recorded for two courses 
each of which was conducted early in the program and again in the final serrester of 
the program. A=General Psychology, B=Culturaf Anthropology, C=General Psychology, 3d 
semester, OCuItural Anthropology, 2d semester. 
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Figure 16. Attitudes toward words relating to Anthropology and Sociology Courses 
as recorded by students in the lecture and programmed study sections on the Semantic 
Differential. Graph shows mean scale values. » ■■■ ■ » Programmed Study Section 
o o Lecture Section. 
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concept of the way the course is taught than did the 
lecture group. This was true of their view of the student 
proctor and the course professor. All other concepts were 
considered to be similar for both groups. In the Sociology 
course only two concepts of the twelve showed any significant 
difference between the two groups. One of these was the 
view of the class members in regard to participating in 
research with the lecture group being somewhat more positive 
than the programmed group. The lecture group was also more 
positive than the programmed group on the concept of educa- 
tional research. Student attitude towards all other con- 
cepts were considered to be similar. In both classes one 
can observe that lecturing is viewed as being a bit on the 
negative end of t lie scale whereas the programmed study concept 
is in the center of the scale. 

Attitudes Toward Lecture and Programmed Study Section s \ 

Table 18 shows attitude data for the lecture and pro- 
grammed study sections of five courses. The percent of 
students in each section who rated the survey item as being 
above average for them is included. These data arc shown for 
eleven items of the attitude survey. From the data it can 
be seen that a greater percent of all the programmed study 
groups had more above average responses than did the lecture 
group on five of the survey items. More programmed study 
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students felt that the amount of work in the course was above 
average. They felt that it was necessary to memorize 
more than they usually did in other courses, The programmed 
study group also indicated more often that their mastery of 
the material was above average. They had more frequent 
reports of improved study habits as well as desire to hear 
lectures, A vast majority of the lecture group felt that 
the amount of lecturing should be kept at the present level, 
if not reduced, 
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DISCUSSION 

From our experience with General Psychology it is 
clear that certain variables are available to the teacher 
which allow him to increase the students' level of participa- 
tion and performance. From this work it is evident that effec- 
tive learning can take place in a setting in which the student 
is required to gain much of the information through his own skill 
of reading and study. Our data would suggest that as far as 
performance can be measured in relation to General Psychol- 
ogy through multiple-choice exams that those students who 
gain the information by their own efforts actually were able 
to respond correctly to a significantly larger amount of exam 
items, Even though students feel that more work goes into the 
study of the course when it is offered in a programmed 
fashion, they are aware that their efforts arc paid off both 
in higher academic scores as well as the added personal satis- 
faction which accompanied this academic success. 

From our observations of the program in General Psy* 
chology it would appear that one important aspect of the pro- 
grammed study technique was that it provided a much stronger 
motivational structure for the student than did the traditional 
lecture procedure, The student was encouraged by the very 
nature of the program requirement to complete all units, to 
continue his study efforts in a systematic, day-to-day basis 
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if he wished to complete the course. Since the work was 
assigned in small units, it was possible for him to initiate 
study activity several times a week. This is not the usual 
case for a lecture course. Students report that on most occa- 
sions the assignment in a lecture course is not studied until 
just prior to the exam. In this case the amount of material 
which must be studied makes it very difficult for the student 
to learn it adequately. Also the actual number of hours 
available for study is limited. The programmed study tech- 
nique seems to gain much of its power from the fact that it 
aids the student to schedule his study time in a more effec- 
tive fashion. 

The Importance of the Course Procton 

During the three semesters of work the proctor was given 
an ever increasing role of importance in the General Psy- 
chology program. In each case both the proctor and the stu- 
dent recognized this improved s igni f icancc . For example, in 
the first semester the proctor was little more than a clerk who 
scored the students 1 quizzes and handed out the next study assign 
meat. The students viewed this proctor role as one which 
added nothing to their enjoyment of the course or to their aca- 
demic success in the course. 

In the second semester the proctor was more personally 
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identified with the student* This was done by assigning each 
student to a team of two proctors. Each team of proctors had 
about fifteen students e This allowed the student to learn to 
know his proctor and to have one or two persons whom he could 
personally identify as responsible for his progress in the 
course* This slight change in the proctors r roles brought a 
bit more recognition to them from the students. The proctors 
also had a slight shift in their opinion as to their contribu- 
tion to the students r enjoyment and success in the course, 

The third semester of General Psychology brought about 
the biggest shift in both the proctors view of his contribution 
to the course and the students 1 recognition of the proctor as 
an essential aspect of the course. 

In this semester the proctor was given six students for 
whom he was totally responsible. He gave the student his 
assignment and study questions, evaluated his preparation with 
the quizzes made available to him, and made judgements con- 
cerning the students' understanding of the material in those 
cases where the students' answers were not the expected answers. 

If the student raised questions that the proctor could not answer, 
or difficult if decisions concerning the students' performance in 
the evaluation session were encountered, the proctor directed 
the student to the instructor. 

In this new role the proctors had very positive feelings 
concerning their contribution to the course. They personally 
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felt that they contributed much to the students 1 academic 
success and enjoyment of the course. They felt that the stu- 
dents they proctored also recognized them as valuable contri- 
butors to the program* 

The improvement in the program brought about by the 
proctors was seen most clearly in the students 1 attitude 
toward the program* Not only were the student proctors able 
to help the students in developing their skills in regards to 
the subject matter of psychology, but also their presence 
made the activity of quiz taking a much more desirable 
activity. Students and their proctors became friends, This 
made the quiz session less of a threat and more of a social 
event. In this more relaxed atmosphere discussion of the 
subject matter became more of a topic of interest than a topic 
of necessity. 

The proctor who is given a role of responsib il ity in the 
program gains in two important ways for himself. First, he 
gains great motivation for learning the course material well, 
and second, he gains a sense of value and enjoyment from helping 
his fellow students. The inves tigators feel that both of 
these outcomes are important to the educational experience of 
the college student. For this reason it is felt that the 
position of proctor is a valuable educational experience for 
which the student might receive college credit, 

‘*7 
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Programmed Courses In Non-Psychology Fields : 

From our data one can conclude that the programmed study 
technique has utility outside the field of psychology. The 
programs in this study were not uniformly successful outside 
the Psychology Department as they were inside the department. 
Based on our observation we felt that the lack of success in 
other departments was not a function of the general instruc- 
tional technique, but rather a function of teaching others in 
the use of the technique. 

All instructors understood the value of short assignment 
length in the program, They all provided programs which met 
what was thought to be an adequately short work unit except 
for the first semester of the Religions of the World course. 
The excessive length of these units was made necessary 
because of a lack of the needed quiz questions for the program 

Quiz preparation was carried out to the satisfaction of 
the authors by all instructors. The items were adequately 
correlated with the assigned reading material and were of a 
difficulty level in most cases which made passing them possibl 
if the material had been studied c 

All instructors understood the incentive value of some 
type of extra point system to be associated with the passing 
of unit quizzes, Not all of these instructors wished to use 
these systems. They did not wish to entice their students 
into taking the quizzes if they did not desire to take them, 
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It is felt that the type of point systems being encouraged by 
the investigators would not have made big improvements in the 
programs if all had used them. In all programs which used 
points but did not require passage of the units of work to 
move on in the program the participation gradually decreased 
as the semester progressed ( table JJ, lines 1,4,8,10,12). The 
points may contribute in part to the motivational aspects of the 
system, but are not strong enough to overcome other competing 
activities in which the student becomes involved. 

One of the important motivational variables not clearly 
understood was the necessity for requiring the student to pass 
the study units in order to gain access to the hourly exams. 

In all but one program outside the Department of Psychology 
this requirement was not specified. The requirement most 
often made was that the student should take the unit quizzes, 
but no consequence was provided for not taking them. At worst 
the student did not earn the extra points which were offered 
for taking the quiz. The importance of this requirement can 
be inferred from Table 2 in which the performance of the first 
and second semester of Cultural Anthropology shows marked 
differences in favor of the group having the requirement to 
take and pass all work units. The change in participation 
for these two groups can be observed in table It is felt 
that if the requirement of taking and passing all units was 
O used in all programs they would have shown some overall 
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improvement in performance and participat ion* 

Student Attitudes Toward The Programmed Study Method ; 

One of the dominant attitudes seen in our study was that 
the programmed study technique required more work to complete 
the course than did the lecture course. This work came about 
not because the assignments were larger but because a greater 
proportion of the assignment was actually completed. The 
definition of what constituted completion of the assignment was 
somewhat more stringent for the programmed study group than 
it was for the lecture groups. In order to do adequate work 
in the programmed study group, the material had to be learned 
well enough to answer the quiz correctly. To learn a few 
general concepts would not be good enough for this programmed 
study method, even though it may be quite adequate 
for the lecture section. Thus the report of more work would 
seem to be quite justified. 

The feeling that the programmed study method put more 
pressure on the student was also a common response of students 
answering the ques tionnaire , The pressure seemed to be assoc- 
iated with the requirement of completing the assignment, that 
is, the student always had some work which had to be done, 
whereas in many of his regular courses the feeling that the 
work had to be done could be more easily escaped. The pres- 
ure was never identified as being uncomfortably great, but was 
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recognized as being present. 

There was some agreement among the students in the var- 
ious courses that mastery of the study matter was improved by 
taking the course in programmed study format. Their more 
active role in the learning process would partially account 
for this increased mastery of material. 

A large number of students reported that one major ad- 
vantage of the course was that they were allowed to progress 
at their own rate through the course, When their schedules 
became crowded at one period of the semester, they could slack 
off in the programmed course and pick i t up when they had more 
time » 

Another point of enjoyment for many students was not 
having to attend three lectures each week. Not all students 
were happy with the fact that lectures were not available, but 
the majority of the students were pleased that they did not 
have to attend regular class meetings. From the responses of 
the students it would appear that they sense inefficiency in 
the lecture procedure. Most students expressed the opinion 
that in the majority of lecture classes it is worthwhile to 
sec a professor in action on some occassion, but to have this 
occur repeatedly becomes tiring and at many points within the 
lecture period becomes redundant and somewhat boring. 

One of the major dissatisfactions with the programmed 
study method of instruction was that the students felt that it 

Pi 
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required a great deal of memorization of material. They felt 
that the quizzes which accompanied the units of work asked for 
specific details which could only be learned through close in- 
spection of the material and memorization of this material 
They felt that this memorization of relatively unimportant 
material was unnecessary. It would be the observation of the 
authors that the same kind of memorization is required in lec- 
ture course as well, In the lecture format the student con- 
centrates on memorizing prior to the date of the exam, whereas 
in the programmed study method memorization is carried on much 
more regularly, It would appear that the student would be 
better satisfied if the material to be learned was set into a 
more experience oriented context which would aid him in remem- 
bering materiale One way to partially remedy this concern 
for memorization is to give more focus tr» the material to be 
learned by use of study guide questions= If the instructor 
makes it clear to the student that lie need only learn what 
the instructor asks him to learn through the study guide 
questions, then he is not at a loss when trying to pick out 
the materials to be remembered. Much of t lie feeling 
concerning memorization, it would appear, arises from the fact 
that it is difficult to decide exactly what should be memorize^ 
Students in both the lecture sections and the programmed 
study sections indicated that the instructor personally did 
O not influence them a great deal. From our survey it was 

no 
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indicated that students in the lecture section felt like the 
instructor influenced them a bit more than in the programmed 
section but in both cases this degree of influence was much 
lower than one initially may have anticipated. 

Additional attitude information was gained by use of the 
Semantic Differential. The concepts to be evaluated were 
12 items which related to the course; the way it was taught, 
student's evaluation of his study in that course, and student's 
response to concepts relating to educational research. The 
actual headings used can be seen in Figure /6. 

One might interpret that even though t lie programmed 
study technique in these two classes (Sociology 1 and Anthro- 
pology 1) had not produced significantly better performance 
on the part of the student, it was slightly more preferred as 
a teaching technique. This is a possible interpretation of 
items six and seven on the Semantic Differential data. 

In the Sociology and Anthropology courses the same 
professor taught botli the lecture and programmed sections. 

In the Sociology course the professor was evaluated on the 
Semantic Differential scales about the same in both classes 
in regards to his role as professor. In the Anthropology 
course this was not true, there was a significant difference 
in regards to how students evaluated the course professor. 

This difference can be associated with the different methods 
ERIC of teaching employed, One might generalize from this 



observation that this method of instruction may enhance some 
professors' presentation of the material as viewed by the 
the s tudents . 

Comments Concerning The Assignment Length Variable ; 

The results of this study support the idea that task 
length is a relevant variable in connection with student pro- 
ductivity in our courses. The difference in the work pattern 
of the two groups was predicted from previous research data of 
schedule length. The difference in achievement between the two 
groups was more dramatic than anticipated. It is conjectured 
that the changed study pattern is directly related to the in- 
creased performance levcle The students in the short assign- 
ment group studied more regularly and also studied less mater- 
ial at one time which increases the probability that much of 
this material would be comprehended and remembered, 

Students in both courses (General Psychology 4 and 
Educational Psychology 2) li ad similar views with regards to 
the desirability of their assignment length and their comprehen- 
sion of the material. In both courses the short assignment 
groups felt the assignment to be about the correct length, 

The long assignment groups had a distinctly different view. 

They indicated that the assignments were somewhat too long* 

This same difference in viewpoint between the long and short 
assignment groups was seen with regards to how well they under- 
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stood the basic concepts of the material. The long assign- 
ment groups indicated that they understood the material in 
this course about as well as they understood material in other 
courses* The short assignment group indicated that they 
understood the material somewhat better than average, 

Further work is required to determine what is an optimal 
assignment length for a given type of material. It is our 
feeling that it is the overall effort invested in the task 
which is the important variable in this study. Task length 
is one dimension which increases the student’s overall required 
effort* Task difficulty also could contribute to the amount 
of effort required to complete the task. 

Comments Concerning The Performance -Reinforcement Relationship : 
Investigators of the learning phenomenon have observed 
that the relationship between the activity of the student and 
how it changes his environment are very important. This re- 
lation has come to be called a contingency. 

The effect of this performance- consequent relation 
has been observed both in the animal laboratory and in the 
classroom setting. In the animal laboratory it may mean 
that the rat pushes a lever which changes his environment in 
sucli a way as to make food available. His food deprived con* 
dition is relieved by the presence of additional food. In the 
same way* the student can answer a question in such a way as 
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to bring a smile upon his teacher f s face. This improves the 
student's social environment by making it a more pleasant sur 
rounding for him. This is also experienced by the student 
who upon completing his test finds that he has all of his 
answers correct. By means of this test taking activity the 
student has been able to change his environment to a more de- 
sirable state. 

The teacher is in the position of establishing these 
contingency relationships. It is the teacher who considers 
the requirements which determine when the environment will 
change to a more desirable condition. It is the teacher who 
says to the student that he must get five problems correct 
before he can receive an "A". The nature of this contingency 
statement docs much to control student behavior in the class- 
room. 

An example of the influence of the educational contin- 
gency in regards to student participation is seen in the vol- 
unteer program of tire Sociology class. Students during 
the first two-thirds of the semester were given the opportun- 
ity to take the quizzes and were encouraged to take t lie 
quizzes. As shown in Figure ^ the participation in quiz 
taking was very low prior to the first exam and fell drasti- 
cally prior to the second exam. After the second exam the 
instructor required t lie students to take and pass the remain- 
ing quizzes in order to have access to the third exam. A con 



tingency was established for the rformance - consequent in- 





- 92 - 



teraction of quiz taking and availability of the third exam. 
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As can be seen in Figure the quiz participation prior to 
the third exan increased dramatically and at the same time an 
associated rise in exam score was observed. Performance was 
significantly better on the third exam than it was on the pre- 
vious two exams, The educational contingency appeared to 
have an important influence not only on participation but also 
on student performance, Since the skills needed to meet the 
above contingency overlapped with the skills needed to perform 
well on the exam, exam performance indirectly benefitted by the 
establishment of this performance-consequent relationship. 

Consideration should be given to the type of response 
required of the student in order to change the environment. 

If th c teacher makes the statement that the student must take 
and pass all the quizzes, there may be several possible ways 
of responding on the part of the student which will meet this 
requirement. Some of these methods of responding may not be 
t lie ones which the teacher had in mind. For example the 
teacher may not have considered the student r s copying answers 
off of some other student's work or the student's having access 
to the quiz prior to taking it, When the teacher makes the 
contingency statement, it is important that he know what re- 
sponse he desires, and accordingly set up the program to 
obtain this desired response. 

Our program was interested in developing two types of rc- 
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sponses. One of these responses related to the student's care- 
ful reading of the text materials assigned and the other re- 
sponse related to the skills of demons trat ing his acquired in- 
formation. The student demonstrated his acquired skills in 
two ways; one by a choice response to a mult iple-choice question , 
and a second by a verbal response to a written question* It was 
felt by the investigators that the two general classes of re- 
sponses were essential for success in the college setting as 
well as for satisfactory competition in the profession in which 
the student becomes engaged after school: Beth of these re- 

sponses arc related to passing the quiz. The oral and choice 
responses to a question which are part of t lie quiz taking pro- 
cedure arc more directly related to passing the quiz, whereas 
the careful reading and studying of the assignment are only 
secondarily related to this final event of quiz passing. 

In developing the contingency it is important that one consid- 
er what behaviors are required in order to accomplish the end 
performance. These performances which bring the student 
to the final consequence arc the performances which the teacher 
wants the student to acquire. It is only after he lias acquired 
these drills that the teacher is willing to say that the stud- 
ent has learned the material or skills desired* 

The next step in this analysis of the contingency re- 
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lates to the consequence which is provided to the student for 
his work. There are several consequences related to the chain 
of events described above. Some of these are more readily 
measured than others. The one consequence which has been 
most readily measured in our study relates to quiz passing. 

With the quiz passing data we wished to learn more about the 
possible relationship existing between quiz passing and the 
maintenance of activities which made up the chain of behavior 
described above. This descriptive data consisted of the com- 
parison of the percent of success (quizzes passed out of 
quizzes taken) as compared to percent participation (quizzes 
taken out of quizzes available). If quiz passing acted 
as a reinforccr, then the student who passed a high per- 
centage of the quizzes should be more inclined to participate 
in the total program. The correlation of data from eight 
programmed courses showed a significant relationship between 
quiz passing and program participation. This suggests that 
passing quizzes possibly acted as a consequence which main- 
tained a chain of events which terminated in this act of pas- 
sing the quiz. 

Quiz passing would maintain its value to the student for 
several reasons. First, quiz passing gave the student extra 
points toward his final grade. Second, students noted that 
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those who passed the quizzes also tended to get higher exam 
scores, This was most readily seen when the students were 
part of a course in which half of the class was being taught 
by a lecture procedure. Third, passing the quiz provided a 
sense of success in that it allowed the students to continue 
in the program. 

Other consequences in this program which would logically 
seem to be effective but for which data is not collected 
easily would be completion of study unit. Task completion 
follows the activity of reading and would act as a rcinforcer 
for reading. From common observation one finds many people 
saying that they feel a sense of relief or satisfaction when 
they have completed an assignment, thus it would be our 
guess that the completion of the assignment acts as a 
reinforcer for this activity of reading. 

Another consequence which would seem to be available 
particularly in those sections where the proctor took a very 
personal interest in the student would be the social contact 
between the proctor and the student. This reinforcer is 
gained at two particular points. One would be the student 
arriving at the room for quiz taking. His actual 
activity of coming is rewarded by his meeting some* 
one who is personally interested in him and his progress in 
tho course* This social contact rcinforcer is also given at 
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the point where the quiz is corrected and the student is told 
whether he has passed or failed. The social contact reinfor- 
cer also is available in the interaction between the proctor 
and student when the quiz taking is oral. This is one reason 
why we feel that the proctor who takes a personal interest 
in a small group of students provides additional social rein- 
forcers in this educational chain of events. 

One final rcinforcer which students describe is that of 
schedule flexibility which the programmed course gave the stu- 
dent to arrange his schedule of activities. This reinforcer 
seems to be gained because the student can escape the aver- 
sivcncss of having great amounts of work that must be completed 
at one particular time within a semester or a week. A stu- 
dent can avoid this work load by either doing the work ahead 
of time in the programmed course or delaying the work until 
some more opportune time, In either case the aversiveness of 
excessive work is escaped, 

Two other aversive consequences were apparent in our 
programmed courses, One of these related to time needed 
to complete the course if several forms of the quiz were 
repeatedly taken. It was thought that the student would find 
most desirable that condition in which he had to come only 
one time in order to pass his quiz. For the student who 
came three tines in order to pass a unit of work much 
effort was required on his part. This added effort would 
encourage a student to pass a quiz on the earliest form 
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The third form of the quiz in the third semester ?f 
General Psychology was completion of f i lling- in- the- 
blank questions in the review book provided by those marketing 
the text. The student was required to answer all the items 
correctly with the aid of his book. Most students required 
to do it found this task mildly aversive. This is thought 
to be another variable which contributed to students 1 pas- 
sing earlier quiz forms. 



The Effect of Performance Evaluation Techniques : 

From observation of student study behavior during this 
year and one-half of investigation of programmed study, it be- 
came clear that how one evaluated the student’s work makes a 
difference in the way the work is done. Our observation was 
basically through student report and instructors questioning 
of students as to their preparation techniques. From these 
informal inquiries several hypotheses were raised regarding the 
role of evaluation in an instructional program. At least two 
roles of evaluation became evident as one considered this pro- 
blem. Evaluation generally is directly related to the pro- 
cess of assigning a grade to students 1 work, This role 
evaluation required that the measuring instrument be reliable. 
When evaluated the student was accurately placed on a scale 
which compared him to other students who also took this class. 

Another role of evaluation particularly associated with 
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the programmed approach of instruction views evaluation as 
leading to a consequence for the students performance. Eval- 
uation as a consequence is also concerned with reliability of 
measurement , but not in the same way as those who use evalua- 
tion as the major way of assigning grades. As a consequence, 
the instructor is interested only in differentiating those who 
have performed to the minimal standard from those who have not 
done so. The instrument must reliably differentiate this point 
but need not differentiate between possibly the hundred dif- 
ferent points of a percentage continuum. To gain reliability 
in the second pass/not pass situation is a somewhat easier task. 

In using evaluation as a consequence the instructor still 
finds that the student's work must be assigned a grade. This 
grade may be designated by pass/not pass or it may be desig- 
nated by some letter or number designation. Tiiis grade has 
usually been arrived at in the programmed study format by 
assigning some value to each of the work units completed thus 
having the grade determined by the total number of units com- 
pleted. The programmed study procedure emphasizes success. 

It emphasizes repetition of material until minimal standards 
are xhieved and assigns a high grade to all who complete the 
required work. 

This method of evaluation effects some students in the 
class differently than others. Students who ordinarily under 
traditional method of grading had received M A M, s will still 
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be able to achieve "A ,M s with the same relative ease they 
have always achieved "A 1H Se Under the procedure where a 
particular minimum performance is required, the "B" student 
will also be able to make an "A" with relative ease. Students 
who are most dramatically effected by this procedure are those 
who fall in the low "C" range or possibly the M D" range. 

These students on previous occasion could receive credit for 
courses without completing much quality work. Many of these 
students, from our observation, are not students who study 
regularly. For these students life is made somewhat more 
difficult. Credit for the course is not available for this 
minimum amount of work. For them to receive a low grade a 
great deal of work is demanded. The programmed course 
requires the student to pass a percentage of the units to get 
credit for the course. Thus the person who wants to just 
"sneak through" t lie course is trapped into the situation of 
putting out just a little more work and receiving a decent 
grade for the course. Some of these students appear to 
be happy that they are put into this situation, others find 
it an infringement upon their rights if not allowed to be 
unproduct ivc. 

The program initially started with all mult iple- choice 
tests. This testing form was chosen because large numbers 
of questions were needed to carry out the evaluation necessary. 
Many texts being used had available pools of these questions which 
could be used for quizzing. Mult iple -choice tests also seemed 
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desirable because they could be graded easily and the student 
could have immediate feedback on his quiz performance# As 
time went on, it became obvious that many disadvantages of the 
multiple-choice test could be overcome by some other form of 
testing. One basic disadvantage discovered was that students 
searched through the material to find those items which would 
most easily lend themselves to multiple-choice questions. Thus 
their study behavior became one of finding possible test items 
rather than gaining a more integrated view of the concepts 
being discussed in the text. 

From our observation of the programming procedure it 
becomes apparent that the technique of evaluation docs much 
more than just assign a grade to the student's performance. 

It is obvious that the student's performance is not ar> inde- 
pendent from the instructional system of which he is a part 
as previously had been thought. The student's ability and 
his motivational level arc in part determined by the education- 
al system itself. Evaluation becomes one of the major 
motivating influences in the programmed study procedure. 

In our evaluation procedures we have shifted from com- 
pletely multiple-choice quizzes to a combination quiz composed 
of both mult iplc -choice and essay question The essay ques- 
tions added as part of the quiz were taken from the 
study guide questions, The essay questions were added to 

the qiiz for two reasons (1) they encouraged the stu- 
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dent to study the material in such a way as to comprehend the 
general concepts and be able to express them in a clear and 
concise manner, and (2) they aided the student by giving him a 
specific focal point for his study* The number of essay questions 
provided as part of the study guide were six to ten and the 
number of the questions asked on the quiz were two. Having 
the questions ahead of time allowed the student to focus his 
attention on particular concepts rather than having him search 
for those items which might be important. The program is more 
interested in developing the skills involved with comprehending 
the material than developing the skills of searching through 
the text to find those concepts which the instructor designate 
as important. 

From our work we were pleased with the essay exams, they 
added a great deal to the effectiveness of the quizzing pro- 
cedure and gave direction to the study behavior of the student. 

On many occassions these essay questions became t he central 
item for discussion between the proctor and the student. As 
part of the quizzing procedure the student was often asked to 
elaborate his answer on his essay question as a means to en- 
courage more interaction between the student and the proctor 
regarding the subject matter. 

Jack Michael, who has used self-study techniques 
for several years, has used only essay exams. From Michael’s 
experience, he feels that extensive use of the essay exams im- 
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proves the student's study procedure as well as the student's 
ability to express himself in relationship to the subject mat- 
ter. Michael's technique is to provide extensive study 
guides directing the student's attention to all the con- 
cepts which he is interested in having them learn. This 
allows a student to give his full attention to the actual 
learning of the concepts rather than searching for and sorting 
out those concepts which might be important to the instructor. 
If the available manpower can be located to grade these exams 
it appears that essay exams are a preferable technique in that 
they develop the skills which are considered valuable and impor- 
tant in the college setting, skills such as: (1) comprehension 

of concepts, gained by studying the material, and (2) self- 
expression, displayed in feeding back the concept learned. 

Public Acceptance of the Programmed Study Met hod » 

The college administration, though initially concerned 
about student reaction to the new method of instruction, was 
willing to go along with the programmed study procedure for a 
semester's Pilot Study. They were sensitive to any com- 
plaints which students made concerning the method. These 
complaints were few, but frequent enough to keep the adminis- 
tration inquisitive about student acceptance of the project. 
After the first semester t he administration allowed 
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the ecperimenters to plan an expanded research program and 
to apply for funds in support of the proposed research, After 
the programmed method was used for two semesters, the adminis- 
tration vas fairly confident that the program could contribute 
to the educational experience of the students thereafter supported 
our efforts without reservation* 

Faculty members on the college staff had various re- 
actions to this new method of instruction. Some showed in- 
terest in the method and were willing to try the procedure in 
their own classes. Other members of the faculty were con- 
cerned that the new teaching procedure required so much stu- 
dent time in preparation that tiie students were not able to do 
adequately their work in other classes* Die majority of the 
faculty members had no strong opinion about the programmed 
study procedure, 

The initial student reaction was one of curious skep- 
ticism. They were interested in a new approach to instruc- 
tion, They especially liked the fact that Bridgewater Col- 
lege was trying something new, but they were not sure they 
liked the new role of the teacher e The students felt that 
the instructor was not doing his job if he was not lecturing, 

student opinion polorized at the end of the first 
year during which the programmed study method was used, The stu- 
dent newspaper carried articles which reflected dissatisfac- 
tion with the programmed study method- The following quotes 
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are taken from the student newspaper: 

"there is not enough time to take the readiness 

quizzes ," 

"there is so much cheating going on that it's really 

of no benefit to very many people." 

"is anyone getting his money's worth?" 

— -"has anyone seen his prof, lately?" 

"have your student-professor relations improved?" 

"if one wanted this type of education all one would 

have to do is send away for a guide and the only 
expense would be postage and a match pak?" 

Other articles by students defended the procedure. 

These students felt that the instructional method was a step 
forward in teaching technology. 

"B.C. seems to have reversed their conservative 

trend, and really giving the students a chance to 
'do their own thing'." 

"many (students) thought iv (programmed course) was a 

good idea in the fact that they didn't have to sit in 
class and listen to boring lectures." 

"another plus for the new system is that the student 

can pace himself and adjust his studying schedule 
around other pressing matters " 

By the beginning of the second year of the project 
students had cone to publicnlly voice acceptance of the 
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procedure, At present the technique is no longer a topic of 
concern. Many students react positively toward the method 
and wish to be part of courses which use the new 
instruct ional me thod. 
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APPENDIX A 



QUESTIONNAIRE FOR ASSESSING STUDENT OPINION 

We are interested in knowing how you feel about several aspects 
of this course which you are now completing. Our desire is to provide 
courses which are interesting, stimulating, and viewed as valuable by 
you. Part of this interest and stimulation is provided by the techniq 
of teaching employed, Your honest opinions are needed so that we may 
refine our procedures and materials. 

Please punch the appropriate letter (a, b, c, d, or e) before you 
choice of the alternatives presented. 

Please punch your student number and card no, "1". 

Thank you 



1, omit 
2* omit 

3. You are taking the opinion survey in which course? 

A, General Psychology 

B, Educational Psychology 

C, Sociology 20 

D, Religions of the World 

E, Social Psychology 

4. What teaching method is being used in this class? 

A. lecture procedure 

B, programmed procedure 

5. In comparison with other courses, I think that the amount of work 
required by this course is: 



A. 


much greater 


D, 


less 


B. 

C. 


greater 

about the same 


E. 


much less 


In 

the 


comparison with other courses, 
work of this course was: 


the 


degree of pressure on me 


A. 


much greater 


D. 


less 


B. 

C. 


greater 

about the same 


E« 


much less 
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7. In comparison with other courses, the percentage of my mastery of 
the assignments in this course was: 



A. much greater 
B* greater 
C. about the same 



D. less 
E* much less 



8 • In comparison with other courses, the feeling of achievement gener- 
ated by passing tests in this one was: 



A a much greater 

B, greater 

C, about the same 



D* less 
E. much less 



9* In comparison with other courses generally, my enjoyment of this one 
was : 



A. much greater 

B. greater 

C. about the same 



D* less 
E, much less 



10* In comparison with other courses generally, the likelihood of cheat- 
ing was : 



A. much greater 
B* greater 
C. about the same 



D* less 
E* much less 



11. In comparison with other courses generally, the temptation to cheat 
was : 



A, much greater 

B, greater 

C, about the same 



D. less 

E, much less 



12. In comparison with other courses generally, my understanding of basi 
concepts and principles in this course was: 



A. much greater 

B. greater 

C. about the same 



D. less 

E, much less 



13. In comparison with other courses generally, my tendency to memorize 
details in this course was: 



A. much greater 
B ' greater 
C. about the same 



D, less 
li e much less 
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14. In comparison with other courses generally, the influence of the 
instructor on me in this course was: 



A. much greater 

B. greater 

C 9 about the same 



D. 

E. 



less 

much less 



15* In comparison with other courses generally, the recognition of me as 
an individual in this course was: 



A. much greater 

B. greater 

C. about the same 



D* 

E. 



less 

much less 



16, As the term went on, I found that my study habits in this course were 



A, greatly improved 

B, improved 

C, unaffected 



D. harmed 

E. greatly harmed 



17. As the term went on, confidence in my ability to master the study 
assignments ; 



A. increased greatly 

B. increased 

C. was unchanged 



D, decreased 

E. decreased greatly 



18. The size of the study assignments in this course was 



A, much too great 

B, too great 

C, about right 



D. too sirall 

E, much too small 



19, As the term went on, my worry about my final standing in the course: 



A. increased greatly 
B * increased 

C, remained about the same 



D, decreased 

E. decreased greatly 



20. As the term went on, my desire to hear lectures: 



A, became much greater 1), 

B, became greater E. 

C, remained about the same 



became less 
became much less 



21, Of your total study time per week, approximately what percentage of 
this time do you devote to studying material in this course? 
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A, 

B, 

C, 



0-151 

15-301 

30-451 



D. 45-601 

B. more than 601 of the total 
study time 
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22. Where do you study? 

A. my own room D. library 

B o dorm study room E. other 

C. "Eyrie” 

23 • Under what conditions do you study? 

A. in silence 

B. usually with a record player* or radio 

C. other 



24. Do you frequently study the course material with some one else? 

A. no D. yes, with 3 other persons 

B. yes, with 1 other person E. yes, with 4 other persons 

C. yes, with 2 other persons 

25, Would you recommend the subject matter of the course to any of your 
good friends? 



A. 


definitely not 


D. 


yes 


B. 

C. 


maybe 

probably 


E. 


no opinion 



26. Which one of the following procedures was most valuable to you in 
preparing for exams? 

A. unit study guide D. group discussion 

B. syllabus 6 course outline E, study hall-quiz period 

27. What was the most interesting aspect of tins class? 

A* textbook D, small group discussions 

B. outside reading E. other (lab observation, etc.) 

C. lectures 
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SEMANTIC DIFFERENTIAL TESTING INSTRUMENT 

The Semantic Differential test was administered with four 
IBM Port-A-Punch cards as shown below. Each card contained three 
concepts to be evaluated on the ten scales shown on the card. 
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Charting Student Performance Rate 




In the chart above the performance of a student on the readiness 
tests is shown. Each day a student took a readiness test, his proctor 
plotted the results. The chart above and on the following two pages 
are from the General Psychology class; 5, d, and 3 points were given 
for Dassing fora A, B and C of the readiness test respectively. The 
points given are recorded in the second colon on the chart; a cumulative 
total is recorded in the third col The first 17 readiness tests 
counted one hourly exam; tr.o second 17 a second hourly exam. Tne student 
shown in the chart amove was pressing to complete the course before the 
Christmas Recess. On ton days he took two readiness tests; three on or.e 
day. The days for taking the six hourly exams are not indicated or. the 
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In the chart above the performance rate of a student is shown who 
had some difficulty with the readiness tests. This student found it 
necessary to take seven form 3 and four form C readiness tests. Though 
tiii s delayed him, he was able to complete on time by taking more than 
one readiness test on ten different days. 
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In the chert above the performance of a student is shown who was 
prone to drop work on his Psychology Course in favor of other demands on 
his tir.e. Four different tiir.es during the semester he took no readiness 
tests for an entire week. By making two or three at a time he was able 
to complete the course on schedule. The top students frequently reported 
that they really liked this flexibility in the utilization of their time. 
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APPENDIX D 



DEVELOP ME MT OF A PROGRAMMED COURSE 
Preparation of Materials : 

1, Choose reading material which can be readily compre- 
hended by the student, 

2, Divide materials into short (10-15 page) assign- 
ments , 

3, Prepare study guides and study questions for each 
uni 1 1 

4* Prepare two or three forms of the quiz for each 
unit. Quiz should be 10-15 items, having several 
essay questions , 



Course Proctors ? 

1* From the pro - regis trat ion list for your class choose 
good students whom you feel can work with other 
students * (Interview each student to be sure you 
have the type of individual you desire for the job.) 
Choosing proctors should be done several weeks 
before class begins, 

2, Provide some incentive for the proctors* work. 

(Course credit, money, etc,) 

3, Assign each proctor five or six students for whom 
he is responsible. 

4, Have regular meetings (weekly) with the proctors 
to discuss the course materials, their student's 
progress, and the general operation of the programmed 
study system. 

Program Operation ; 

1. Provide meeting rooms and tines for the proctor and 
his students to meet, 

2. Proctors provide study guides for each unit to be 
studied, 

3. Proctors administer quizzes to students and grade 
answers . 
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4, Proctors discuss essay questions with student and 
answer any questions the student has regarding the 
subject matter or course procedures. 

5, Proctors chart the student's progress on the stu- 
dent's individual cummulative graph. 

6, Instructor provides and grades major exams when 
student has finished all work prior to exam. 

Enrichment Activities : 

1, Several lectures or demonstrations can be scheduled 
to add novelty to the students' class work routine. 

2, Discussion groups can be added to provide the stu- 
dent with direct contact with the instructor and to 
provide the student with an opportunity to develop 
verbal skills in the subject matter field. 
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